SMS Spam Classifier
The SMS Spam Collection dataset contains 5,572 messages with an 87/13 ham/spam split — a class imbalance that makes raw accuracy a misleading metric. This project trains two classical models (ComplementNB and LinearSVC) on TF-IDF features with bigrams, compares them against Amazon Bedrock zero-shot classification using Claude Haiku, and deploys the result as a Flask web app. The core narrative is the comparison across three fundamentally different classification paradigms: probabilistic, geometric, and generative. Every prediction from every model is accompanied by a ranked list of the words that drove the decision, extracted directly from model internals rather than a post-hoc approximation.
Quick Facts
Overview
Problem
Spam detection looks simple until you account for class imbalance (87% ham). A model that predicts ham for every message scores 87% accuracy while catching zero spam. The challenge is building a classifier that achieves high recall on the minority class without flooding the inbox with false positives — and explaining its decisions in terms a user can actually verify.
Solution
Trained ComplementNB (designed for imbalanced text) and LinearSVC (standard for high-dimensional sparse TF-IDF) with bigram features capturing phrase-level spam signals like "free prize" and "call now". Compared class_weight settings empirically and kept whichever improved F1. Added a third comparator — Claude Haiku via Amazon Bedrock — running zero-shot on a 40-message balanced sample. Built a word-level explanation module (text_explainer.py) that extracts log-probability ratios from NB and coefficient weights from SVM to surface the exact tokens driving each prediction.
Challenges
Class imbalance required explicit handling — stratified splitting to preserve the 13% spam ratio in both train and test sets, and empirical comparison of class_weight='balanced' rather than assuming it always helps. The Bedrock integration required stripping markdown code fences from JSON responses. Azure App Service quota was zero in eastus across all SKUs; discovered centralus had available F1 quota and deployed there directly, bypassing the B1→F1 portal workaround described in the spec.
Results / Metrics
ComplementNB: Accuracy 97.1%, Precision 87.7%, Recall 91.3%, F1 0.895, ROC-AUC 0.984.
LinearSVC: Accuracy 98.4%, Precision 97.8%, Recall 89.9%, F1 0.937, ROC-AUC 0.988.
Bedrock (40-message sample, zero-shot): 92.5% accuracy, 40/40 calls succeeded.
LinearSVC leads on F1 and ROC-AUC. NB achieves higher recall. Bedrock performs competitively with no training data, at orders-of-magnitude higher latency and cost per prediction.
Screenshots
Click to enlarge.
Click to enlarge.
